• 专利附录
  • 专利说明
  • 权利要求
  • 类似技术
  • 同族专利
  • 引用文献
  • 法律状态
  • 优先权
    • 专利摘要:
    Methods and devices for quickly dimensioning an object are provided. An exemplary method involves recording, by a plurality of cameras, image data representative of an object; analyzing, by a processor, the image data to identify a plurality of candidate angles of the object; detecting, by the processor, a proximity of an appendix to each of the candidate angles; confirming, by the processor, based on respective proximity of the tag to the candidate corners of the object, that a first of the candidate corners is a corner of the object; and calculating, by the processor, based on the confirmed angle of the object, a size of the object.
    公开号:BE1025916B1
    申请号:E20185780
    申请日:2018-11-07
    公开日:2020-02-07
    发明作者:Nicole D Tricoukes；Michael J Giannetta；Matthew L Kowalski
    申请人:Symbol Technologies Llc；
    IPC主号:
    专利说明:

    METHODS AND DEVICES FOR QUICK DIMENSIONING AN OBJECT BACKGROUND
    In a stock environment, such as a retail store, a warehouse, a shipping facility, etc., it is useful to know the dimensions of a box item. Existing imaging systems are able to estimate the dimensions of a box object by identifying angles of an object. However, these existing systems can often take a very long time to identify the real corners of the box object from all possible angles. In addition, dimension errors can occur if real angles are incorrectly identified.
    SUMMARY OF THE INVENTION
    According to the invention, a computer-implemented method is provided, comprising recording, by a plurality of cameras, image data representative of an object, analyzing, by a processor, the image data to create a multiple number of candidate angles of the object identifying, detecting, by the processor, a proximity of an appendix to each of the candidate angles, confirming, by the processor, based on the respective proximity of the appendix to the candidate angles of the object, which is a first of the candidate angles is an angle of the object, and the processor calculates, based on the confirmed angle of the object, a size of the object.
    Analyzing the image data to identify the candidate angles of the object can preferably identify, by the processor, portions of the image data indicative of one or more
    BE2018 / 5780 edges of the object, extrapolating, by the processor, the one or more edges of the object to determine a multiple number of intersections of the edges, and identifying, by the processor, the candidate angles of the object on basis of the intersections of the edges.
    Detecting a proximity of the tag to one of the candidate angles of the object and confirming that the first of the tag angles is an angle of the object can analyze the image data to identify portions of the image data indicative of the tag detecting that the tag has stopped moving for a threshold time period, determining a distance between the stopped tag and each of the candidate angles, and identifying the first of the tag angles if it has the shortest distance to the stopped tag of the tag user.
    Preferably, the computer-implemented method may further comprise instructing a user to place the tag on a first corner of the object prior to determining the distance between the stopped tag and each of the candidate angles.
    More preferably, the computer-implemented method may further include, after confirming the angle of the object, instructing the user to place the tag on a second angle of the object.
    The computer-implemented method may further include detecting, by a sensor, a signal transmitted by a transmitter associated with the user's tag, the signal being generated in response to a user's tactile action.
    Detecting the proximity may preferably detect a first proximity of the tag to one of the candidate angles in response to detecting the signal that is
    BE2018 / 5780 broadcast by the transmitter associated with the tag.
    Calculating the size of the object can, for example, comprise determining a length of an edge of the object.
    Calculating the size of the object can additionally or alternatively analyze image data to identify a reference mark associated with the object, the reference mark having a known size, and calculating the size of the object can be based on include the known size of the reference mark.
    Calculating the size of the object can also determine, based on the angle of the object, a length of each edge of the box object, and determining, using the lengths of the edges, a volume of the object .
    According to another aspect of the invention, a system is provided comprising a heads-up display assembly comprising a presentation generator and a head mount, the heads-up display assembly being configured to be worn by a user, a plurality of cameras are attached to the headsup display assembly, the cameras being configured to record image data associated with an object and a user's appendage, a memory configured to store computer-executable instructions, a processor configured is to interact with the presentation generator, the plurality of cameras, and the memory, and is configured to execute the computer-executable instructions to prompt the processor to analyze the image data associated with the object to identify multiple number of candidate angles of the object, detecting v a proximity of the user's appendix to a
    BE2018 / 5780 first of the candidate corners of the object, to determine, based on the proximity of the appendix to the first of the candidate corners, whether the first of the candidate corner is a real corner of the object, and when the first of the object candidate angles is a true angle, calculating a dimension of the object based on the first of the candidate angles.
    The computer-executable instructions may prompt the processor to, when executed, identify the candidate angles by identifying data indicative of one or more edges of the box object, extrapolating the one or more edges of the box object, and identifying the candidate angles of the box object based on the intersections of the extrapolated edges.
    The computer-executable instructions may also cause the processor to detect the proximity of the tag to the first of the candidate angles by detecting that the tag has stopped moving for a threshold time period, and in response to detecting that the tag is for the threshold time period has stopped moving, determining a distance between the stopped appendage of the user and the first of the candidate angles.
    The computer-executable instructions may further prompt the processor to instruct the user to place the tag on a first corner of the object prior to determining the distance between the stopped tag and the first of the candidate angles.
    The computer-executable instructions may further prompt the processor to instruct the user, after determining that the first of the candidate angles is a true angle of the object, to place the tag on a second angle of the object.
    BE2018 / 5780
    The system may preferably include a sensor, wherein the instructions executable by the processor may cause the processor to detect, via the sensor, a signal transmitted by a transmitter carried by the user, the signal being generated in response to a user's tactile action.
    The computer-executable instructions may cause the processor to detect the proximity of the tag to the first of the candidate's corners of the object in response to the signal transmitted by the transmitter.
    The instructions executable by the computer may also cause the processor to calculate the size of the object by determining a length of an edge of the object.
    For example, the computer-executable instructions may prompt the processor to analyze the image data to identify a reference mark associated with the object, the reference mark having a known size, and the computer-executable instructions may cause the processor to calculating the dimension of the object based on the known dimension of the reference mark and the determined length of the edge.
    The computer-executable instructions may cause the processor to calculate the size of the object by determining a length of an object's edge and determining, using the length of the object's edge, a volume of the object.
    BE2018 / 5780
    BRIEF FIGURE DESCRIPTION OF THE DIFFERENT VIEWS OF THE DRAWINGS
    The accompanying figures, wherein like reference numerals refer to identical or functionally similar elements in the separate views, together with the detailed description below, are incorporated into and form part of the description and serve to further illustrate embodiments of the concepts described herein. , and explain various principles and benefits of those embodiments.
    FIG. 1A is a block diagram of an exemplary HUD assembly constructed in accordance with some embodiments.
    FIGS. 2A and 2B illustrate an exemplary HUD assembly that the exemplary HUD assembly of FIG. 1 can implement in accordance with some embodiments.
    FIG. 3 illustrates the exemplary HUD assembly of FIGS. 2A and 2B attached to a user's head in accordance with some embodiments.
    FIG. 4 illustrates exemplary light engines attached to an exemplary HUD assembly in accordance with some embodiments.
    FIG. 5 illustrates a user carrying an exemplary HUD assembly to which cameras are attached with the user looking at an exemplary box object to be dimensioned in accordance with some embodiments.
    FIG. 6 is a block diagram representative of an exemplary logic circuit in accordance with some embodiments.
    BE2018 / 5780
    FIG. 7 is a schematic diagram of an imaging system that uses an HUD assembly in accordance with some embodiments.
    FIG. 8 shows that a bar code is placed on the surface of a box object in accordance with some embodiments.
    FIG. 9A is an example of a screen that would be seen by a user of the HUD assembly when a box object is being dimensioned, the example showing a recorded image of the box object with the geometric sign on one side of the box object, in accordance with some embodiments.
    FIG. 9B is an example of a screen that would be seen by a user of the HUD assembly when a box object is being dimensioned, showing the box dimensions and the volume of the box object, in accordance with some embodiments.
    FIG. 10A shows a recorded image of a box object in accordance with some embodiments.
    FIG. 10B shows some edge lines in the recorded image in accordance with some embodiments.
    FIG. 10C shows some points in the recorded image that are possible vertices of the box object in accordance with some embodiments.
    FIG. 11 is a flow chart of a method for determining the dimensions of a box object in accordance with some embodiments.
    FIG. 12 is a flow chart of a method for analyzing image data around a plurality of potential vertices
    BE2018 / 5780 to identify a box object in accordance with some embodiments.
    FIG. 13 is a flow chart of a method for detecting the position of a user tag with respect to potential vertices of the box object in accordance with some embodiments.
    For the sake of simplicity and clarity, elements in the figures are illustrated and not necessarily drawn to scale. For example, the dimensions of some of the elements in the figures may be exaggerated relative to other elements to help improve understanding of the embodiments shown herein.
    The device and method components are represented, where appropriate, by conventional symbols in the drawings, showing only those specific details that are relevant to the understanding of the embodiments shown herein to thereby not obscure the illustration with details evidently clear.
    DETAILED DESCRIPTION
    Systems and methods are provided for quickly dimensioning a box object. The systems and methods may include a plurality of cameras to record image data representative of an object. A processor can analyze the image data to identify a plurality of candidate angles of the object and to detect a proximity of an appendix to each of the candidate angles. Based on the proximity of the appendix to each of the candidate angles of the object, one of the candidate angles is confirmed as an angle of the object. Once at least two angles of the object have been identified and confirmed, the processor can calculate the dimensions and / or volume of the object based on
    BE2018 / 5780 of the real corners of the box object. Advantageously, the real angles of the object can be quickly identified from the candidate angles, greatly reducing processing time and speeding up dimensioning.
    FIG. 1 is a block diagram of an exemplary HUD assembly 100 constructed in accordance with the teachings of this description. Alternative implementations of the exemplary HUD assembly 100 of FIG. 1 comprise one or more additional or alternative elements, processes and / or devices. In some examples, one or more elements, processes, and / or devices of the HUD assembly 100 of FIG. 1 are combined, separated, rearranged or omitted.
    The exemplary HUD assembly 100 of FIG. 1 includes a presentation generator 102 and a main mount 104. The main mount 104 is constructed to attach the presentation generator 102 to a person's head such that a presentation generated by the presentation generator 102 is consumable by the person. The presentation includes visual media components (e.g., images) and / or audio media components. To generate images such as static or animated text and / or graphs, the presentation generator 102 of FIG. 1 an image generator 106. The exemplary image generator 106 of FIG. 1 is in communication with one or more sources of image data. The image data received by the image generator 106 is representative of, for example, text, graphs and / or augmented reality elements (e.g., information that overlaps with objects in the field of view). The exemplary image generator 106 of FIG. 1 comprises exposure units (light engines) 108 which convert the received image data into patterns and pulses of light. The exposure units 108 communicate the generated light to a waveguide 110 such that the images associated with the received data are shown to the user via the waveguide 110. In some examples, the exposure units 108 include optics that
    BE2018 / 5780 condition or manipulate generated light (e.g., polarize and / or collimate) prior to providing the light to the waveguide 110. Although the exemplary image generator 106 uses the exposure units 108 and the waveguide to present visual components of the presentation, exemplary HUD assembly 100 of FIG. 1 use any suitable image generation technique such as, for example, cathode ray tube (CRT) devices or scanning lasers.
    In the example of FIG. 1, the exposure units 108 use a light source (e.g., light emitting diodes [LEDs]) to generate light based on the received data. In some examples, the exposure units 108 receive processed data in a state for immediate conversion to light. In some examples, the exposure units 108 process raw image data before the image data is converted to light. To perform such processing, the exemplary light engines 108 of FIG. 1 and / or the exemplary light engines 108 of FIG. 1 in communication with one or more logic circuits configured to process the image data.
    The exemplary waveguide 110 of FIG. 1 directs the light received from the exposure units 108 in a direction and a pattern associated with the image data. In the illustrated example, the waveguide 110 comprises a plurality of internal surfaces that form a light guide to internally reflect the light as the light propagates from an input to an output. The exemplary light guide 110 includes a grid at the output to bend the light to an eye of the user, thereby showing the image to the user. The exemplary waveguide 110 of FIG. 1 comprises a first and a second lens arranged to be placed over, respectively, a first and a second eye of the user. However, any suitable shape or format is possible for the waveguide 110. In the illustrated example, the waveguide is 110
    BE2018 / 5780 transparent such that the user can see the environment simultaneously with the displayed image, or only the environment when no image is displayed on the waveguide 110.
    The exemplary presentation generator 102 of FIG. 1 includes an audio generator 112 that receives audio data and converts the audio data to sound through a headphone jack 114 and / or a speaker 116. In some examples, the audio generator 112 and the image generator 106 work together to generate an audio-visual presentation .
    In the example of FIG. 1, the example presentation generator 102 includes (e.g., accommodates) a plurality of sensors 118. In the example of FIG. 1, the plurality of sensors 118 includes a light sensor 120, a motion sensor 122 (e.g., an accelerometer), a gyroscope 124, and a microphone 126. In some cases, the plurality of sensors 118 may include a sensor configured to detect signals that are sent by a Bluetooth low energy radio, for example associated with a user's appendix In some examples, the presentation generated by the example image generator 106 and / or the audio generator 112 is affected by one or more measurements and / or detections generated by one or more of the sensors 118. A characteristic (e.g., a degree of transparency) ) of the screen generated by the image generator 106 may, for example, depend on an intensity of ambient light detected by the light sensor 120. Additionally or alternatively, one or more modes, operational parameters, or settings are determined by measurements and / or detections generated by one or more of the sensors 118. For example, the presentation generator 102 may enter a standby mode if the motion sensor 122 has not detected motion in a threshold time amount.
    BE2018 / 5780
    The exemplary presentation generator 102 of FIG. 1 includes a camera subsystem 128. In some examples, the camera subsystem 128 is attached to or carried by the same housing as the presentation generator 102. In some examples, the camera subsystem 128 is attached to or carried by the main mount 104. The exemplary camera subsystem 128 comprises at least two cameras 130 and a microphone 132 for, respectively, recording image data and audio data representative of an environment surrounding the HUD assembly 100. In some examples, the image and / or audio data recorded by the cameras 130 and / or the microphone 132 are integrated with the presentation generated by the image generator 106 and / or the audio generator 112. The camera subsystem 128 of FIG. 1, for example, communicates data to the image generator 106, which can process the image data and to generate one or more associated images on the waveguide 110. In some examples, the image data and / or audio data recorded by the cameras 130 and / or the microphone 132, respectively, are stored in memory 135 of the example HUD assembly 100. In some examples, the image data and / or audio data recorded by respectively the cameras 130 and / or the microphone 132, communicated via, for example, a USB interface 134 from the camera subsystem 128 to a device (e.g., a server or external memory) external to the HUD assembly 100.
    The exemplary presentation generator 102 of FIG. 1 includes a plurality of interfaces 136 that are configured to allow HUD assembly 100 to communicate with one or more external devices 142 and one or more networks 138. In the example of FIG. 1, the interfaces 136 included converters 140 (e.g., an HDMI to LVDS-RGB converter) to convert data from one format to another, a USB interface 144, and a Bluetooth® audio transmitter 146. In some examples, the Bluetooth works ® audio transmitter 146
    BE2018 / 5780 together with one or both of the microphones 126, 132 of the HUD assembly 100 to receive voice input from the user and to transfer the voice input to one or more of the external devices 142. The voice input may be provided, for example to a mobile computing device carried by the user via the HUD assembly 100 using the Bluetooth® audio transmitter 146. Examples of external devices 142 include keypads, Bluetooth® click buttons, smart watches, and mobile computer devices.
    The example image generator 106, the example exposure units 108, the example audio generator 112, the example camera subsystem 128, the example converters 140, the example USB interfaces 134, 144, and / or more generally, the example presentation generator 102 of FIG. 1 are implemented by hardware, software, firmware and / or any combination of hardware, software and / or firmware. In some examples, at least one of the example image generator 106, the example exposure units 108, the example audio generator 112, the example camera subsystem 128, the example converters 140, the example USB interfaces 134, 144 and / or more generally the example presentation generator 102 of FIG. 1 implemented by a logic circuit. As used herein, the term "logic circuit" is expressly defined as a physical device that includes at least one hardware component that is configured (e.g., through operation in accordance with a predetermined configuration and / or through execution of stored machine-readable instructions) to manage one or more machines and / or perform operations on one or more machines. Examples of a logic circuit include one or more processors, one or more co-processors, one or more microprocessors, one or more controllers, one or more digital signal processors (DSPs), one or more application-specific integrated circuits (ASICs), one or more field-programmable gate
    BE2018 / 5780 arrays (FPGAs), one or more microcontroller units (MCUs), one or more hardware accelerators, one or more special-purpose computer chips, and one or more system-on-chip (SoC) devices. Some exemplary logic circuits, such as ASICs or FPGAs, are specifically configured hardware for performing operations. Some exemplary logic circuits are hardware that executes machine-readable instructions to perform operations. Some exemplary logic circuits include a combination of specifically configured hardware and hardware that execute machine-readable instructions.
    As used herein, each of the terms "tactile machine-readable medium", "non-transitory machine-readable medium", and "machine-readable storage device" is explicitly defined as a storage medium (e.g., a hard disk plate) , a digital widely applicable disc, a compact disc, flash memory, read-only memory, random access memory, etc.) on which machine-readable instructions (for example, program code in the form of, for example, software and / or hardware) can being saved. Further, as used herein, each of the terms "tactile machine-readable medium", "non-transitory machine-readable medium" and "machine-readable storage device" is expressly defined to exclude signal propagation. That is, as used in any claim of this patent, a "tactile machine-readable medium" cannot be read as being implemented by propagating a signal. Furthermore, as used in a claim of this patent, a "non-transit machine readable medium" cannot be read as being implemented by propagating a signal. Furthermore, as used in any claim of this patent, a "machine-readable storage device" cannot be read as being implemented by propagating a signal.
    BE2018 / 5780
    As used herein, each of the terms "tactile machine-readable medium", "non-transitory machine-readable medium" and "machine-readable storage device" is expressly defined as a storage medium on which machine-readable instructions are stored for any suitable length of time (e.g., permanently, for a longer period of time (e.g., while a program associated with machine-readable instructions is being executed) and / or a short period of time (e.g., while machine-readable instructions are being cached and / or buffered during a buffer process)).
    FIGS. 2A and 2B illustrate an exemplary HUD assembly 200 that the exemplary HUD assembly 100 of FIG. 1 can implement. The exemplary HUD assembly 200 of FIG. 2A includes a presentation generator 202 and an exemplary main mount 204. The exemplary presentation generator 202 of FIG. 2A accommodates or carries components that are configured to, for example, generate an audiovisual presentation for consumption by a user using the exemplary HUD assembly 200 of FIG. 2A. The presentation generator 202 of FIG. 2A, for example, accommodates or carries the components of the exemplary presentation generator 102 of FIG. 1.
    FIG. 3 illustrates the exemplary HUD assembly 200 of FIGs. 2A and 2B attached to a user's head 300.
    FIG. 4 illustrates exemplary exposure units 402 which, for example, the exposure units 108 of FIG. 1 implement. As described above, the exposure units 402 generate light to be guided through the waveguide. The cameras 402 can be configured to record image data representative of a box object and the hands of a user when a user carrying the HUD assembly 200 looks at a box object. Although the exemplary light engines 402 of FIG. 4 are positioned above each eyepiece
    BE2018 / 5780 lighting units are positioned at any suitable location, such as, for example, on the edges of the frames. FIG. For example, 5 illustrates a user wearing the HUD assembly 200 and looking at a box object, with cameras 502 attached to the sides of the main mount, recording image data comprising the box object and the user's hands.
    FIG. 6 is a block diagram representative of an exemplary logic circuit that can be used to, for example, the example image generator 106, the example light engines 108, one or more example interfaces 136 and / or the example audio generator 112 of FIG. 1 to implement. The exemplary logic circuit of FIG. 6 is a processing platform 600 capable of executing machine-readable instructions to, for example, operations associated with the example HUD assembly 100 of FIG. 1 to implement.
    The exemplary processing platform 600 of FIG. 6 comprises a processor 602 such as, for example, one or more microprocessors, controllers and / or any suitable type of processor. The exemplary processing platform 600 of FIG. 6 includes memory (e.g., volatile memory, non-volatile memory) accessible to the processor 602 (e.g., via a memory controller). The exemplary processor 602 interacts with the memory 604 to obtain, for example, machine-readable instructions stored in the memory 604. Additionally or alternatively, the machine-readable instructions may be stored on one or more removable media (e.g., a compact disc, a digital versatile disc, removable flash memory, etc.) that may be coupled to the processing platform 600 to provide access to the machine-readable instructions stored thereon. In particular, the machine-readable instructions may be stored on the memory
    BE2018 / 5780
    604 include instructions for performing one of the methods described in more detail below with FIGs 11-14.
    The exemplary processing platform 600 of FIG. 6 further comprises a network interface 606 to enable communication with other machines, via, for example, one or more networks. The exemplary network interface 606 includes any suitable type of communication interface (s) (e.g., wired and / or wireless interfaces) that are configured to operate in accordance with any suitable protocol. The exemplary processing platform 600 of FIG. 6 includes input / output (I / O) interfaces 608 to enable receipt of user input and communication of output data to the user.
    FIG. 7 is a schematic diagram of an imaging system
    700 using a HUD assembly such as HUD assembly 100 and / or HUD assembly 200 in accordance with some embodiments. The imaging system 700 in FIG. 7 includes two cameras 702 and 704 attached to or carried by the HUD assembly 200, for example, as shown in FIG. 5. The cameras 702, 704 operate to record the light along respective optical paths / axes 708, 710. In general, the cameras are designed to cooperate to record light scattered or reflected on target object (s) (e.g., box object 712, a user attachment 714) as pixel data over a two-dimensional imaging field of view (FOV).
    In some operating areas, as shown in FIG. 8, a bar code 40 is placed on one or more sides of a box object 30, and the imaging system 700 is used to decode the bar code 40 to identify the box object or to find other information from the box object. In accordance with some embodiments, as the bar code format 40 may
    BE2018 / 5780 (e.g. its width or its length) is known, the volume or the three box dimensions (ie, the length L, the width W, and the height H) of the box object 30 are estimated as the box object 30 and the bar code 40 both are included in an image by the imaging system 700. In general, if the distance between two points on a geometric mark 35 on the box object 30 is known, the volume or the three box dimensions of the box object can be estimated from the image of the box object box object 30 as recorded by the imaging system 700. Examples of the geometric sign 35 include a one-dimensional bar code, a two-dimensional bar code, a logo, a shipping label, or the combination of isolated dot characters printed or the F0, F1, F2 side of the box object 30. Example methods and devices for dimensioning a box object in such a way are described in US Patent No. 9,741,134, filed December 16, 2013.
    FIG. 9A is an example of a display that would be seen by a user of the HUD assembly when a box object is being dimensioned. Specifically, FIG. 9A shows a recorded image of the box object 30 with the geometric mark 35 on one side of the box object 30 in accordance with some embodiments. In this example, the geometric mark 35 is in the form of a rectangular body, and two angular points of the geometric mark 35 are used as two reference points "P" and "Q" on a surface of the box object 30 as shown in FIG. 8. In FIG. 9A, these two reference points in the recorded image are labeled as "P" and "Q". The positions of these two reference points in the recorded image can be automatically identified with the imaging system 50 as described in U.S. Pat. Patent No. 9,741,134. In the illustrated example, a user may touch or point to the reference points to inform the imaging system 50 of the location of the
    BE2018 / 5780 reference points. That is, the imaging system 50 can use the recognizable (e.g. based on a comparison with an image or shape known to correspond to an appendix such as a finger) image of the user appendix to learn the location of the reference points and / or confirm. In some embodiments, more than two reference points are used to perform a calculation (e.g., a volume of the box object) or an algorithm, and the position of one of these more than two reference points in the recorded image can be automatically identified with the imaging system 50 or can be manually identified by the user.
    Similarly, the box object in the recorded image can be identified by the six angles v0, v1, v2, v3, v4 and v5. As described in detail below, the examples described herein use the ability of the imaging system 50 to recognize a tag of a user (e.g., based on image recognition or a recognizable sign carried by the user) at the positions of these six indicate and / or confirm corner points of the box object in the recorded image.
    The confirmed positions of the corner points are used to determine one or more dimensions of the box object. FIG. 9B depicts an example of a display 98 that would be seen by a user of the HUD assembly when the one or more dimensions of the box object have been calculated.
    Specifically, FIG. 10A shows a recorded image of a box object, FIG. 10B shows that some edge lines in the recorded image are determined, and FIG. 10C shows some points in the recorded image (for example, some intersections of the edge lines) selected as possible vertices of the box object. Typically, corner recognition algorithms will generate more candidate corner points than there
    BE2018 / 5780 are actually present on the box object. In the illustrated example, the box object has only eight real vertices. Examples described herein use the ability of the imaging system 50 to recognize a user tag to identify and / or confirm the candidate vertices in the image data as corresponding to real corners of the box object. In some examples disclosed herein, a user is instructed to pinpoint or touch the real corners of the box object to distinguish between the real corner points of the box object and any false candidate corner point of the box object. Advantageously, the user's confirmation of and / or indication for the true corner points of the box object speeds up the processing time of the dimensioning system, because the processors no longer have the time-intensive task of determining which candidate corner points of the box object are true corner points of the box object have to complete.
    FIG. 11 is a flow chart of a method 1100 for determining the dimensions of a box object in accordance with some embodiments. A user wearing a HUD can look at a box object to be dimensioned, for example, as shown in FIG. 5, wherein the box object is effectively placed in the field of view of the HUD.
    At block 1102, image data is included that includes the box object, e.g. via a stereoscopic lens camera mounted on each side of the user's HUD. In some cases, the user may indicate that dimensioning is about to begin, for example via a voice command, a gesture, the push of a button, etc., and the image data comprising the box object may be recorded in response to the user's indication. In some cases, the user can pick up the box object as shown in FIG. 5 while in other cases
    BE2018 / 5780 (for example when the box object is particularly heavy) the user can simply look at the box object.
    At block 1104, the image data comprising the box object is analyzed to identify a plurality of candidate corner points of the box object in the image data, which is described in more detail with respect to FIG. 12. In the illustrated example, coordinates of the candidate vertex points are stored in memory and designated as belonging to candidate vertex points.
    The user can point or touch a vertex of the box object within the field of view of one or both of the stereoscopic lens camera mounted on each side of the HUD. At block 1106, the position of a user tag (e.g., a finger) with respect to the possible angular points of the box object is determined, e.g., by one or more processors, as discussed in more detail with respect to FIG. 13.
    At block 1108, the true vertices of the box object are determined and / or confirmed by one or more processors based on the proximity of the user tag to the candidate vertex of the box object. That is, a candidate vertex close to the user tag (for example, within a threshold distance or the shortest distance among the multiple number of candidate vertexes) is probably a real vertex that the user touches or points to.
    At block 1110, the image data is analyzed to identify a reference mark on the box object. Specifically, the reference mark has known dimensions. The reference mark is, for example, a standard format shipping label on the box object. In a single case, a bar code associated with the reference mark is analyzed, for example, by one or more processors and the barcode can indicate the dimensions of the reference mark.
    BE2018 / 5780
    At block 1112, the dimensions and / or volume of the box object are calculated based on the known dimensions of the reference mark and the determined and / or confirmed true vertices of the box object. Exemplary methods and / or devices for determining the dimensions and / or volume of the box object using the knowledge of the angles and the reference marks are described in U.S. Pat. Patent No. 9,741,134.
    FIG. 12 is a flowchart of a method 1200 for analyzing image data to identify a plurality of candidate corner points of a box object, block 1104 of FIG. 11 is being worked out. At block 1202, image data comprising the box object can be analyzed, for example, by one or more processors. The analysis includes, for example, smoothing such as Gaussian Blur, detection such as Canny algorithm, Hough transform algorithm with contour detection, Convex Hull algorithm, etc. Example methods and devices for using image processing algorithms are described in U.S. Pat. Patent No. 9,741,134. At block 1204, image data indicative of box object edges is identified, for example, by one or more processors based on the analysis of block 1202.
    At block 1206, the box object edges are extrapolated to determine possible intersections of each box object edge with every other box object edge, for example as shown in FIG. 10 C. At block 1208, possible box object angles are identified based on possible intersections of extrapolated box object edges. In general, as shown in FIG. 10 C, a large number of candidate box object angles will be identified by the system, but only eight real box object angles on the box object.
    FIG. 13 is a flow chart of a method 1300 for detecting the position of a user tag (e.g., a finger)
    BE2018 / 5780 with respect to possible angles of the box object, wherein block 1106 of FIG. 11 is being further elaborated.
    At block 1302, a user can be instructed to place a tag (e.g., a finger) on a first corner of the box object. The audio generator 112 of the presentation generator 102 may, for example, provide audio instructions, for example "please touch the top right corner" or "please point to first corner". Additionally or alternatively, the presentation generator image generator 106 may provide similar visual and / or textual instructions.
    At block 1304, image data of the box object and surrounding area is analyzed by, for example, one or more processors. Based on the analysis, image data is identified that is indicative of a user tag at (block 1306). In some cases, the analysis may include identifying a bared tag, for example based on color, shape etc. In some cases, the user may wear a distinctive (e.g. recognizable to the system based on a known image) glove, and are the image data indicative of the user attachment image data indicative of the glove. For example, the glove could be a specific color configured for rapid contrast-based identification, or the glove could include a specific marking or shape that is known to the system at a fingertip of the glove. In some cases, the glove may have a light and / or other visually identifiable indicator that can be identified by analyzing the image data.
    At block 1308, an end in motion of the user tag is detected. For example, if the user tag is identified in the image data and stopped moving during a threshold time period (e.g., five seconds), it indicates that the user is currently at the first corner of the box object
    BE2018 / 5780 touches or points. Additionally or alternatively, in some examples where the user is wearing a glove, the user may press a button or else an ultra-small Bluetooth low energy (BLE) radio that is attached or electronic circuitry printed in the finger of the activate the glove to indicate that the user is currently touching or pointing at the first corner of the box object. That is, the BLE radio can output a signal that is detectable by a sensor 118 of the presentation generator 102 of the HUD 200.
    At block 1310, once the user has indicated (e.g., by stopping, pressing a button, etc.) that he or she is pointing or touching the first vertex of the box object, the distance between the user tag and each of the candidate corner points of the box object determined, for example, by one or more processors. That is, the processor (s) analyze the image data to determine relative positions of each of the identified candidate vertex and the user tag and to determine the distance between the user tag and each candidate vertex.
    At block 1312, the candidate vertex that has the shortest distance to the stopped user appendix, for example by one or more processors, is identified as a real vertex of the box object. That is, the vertex that has the shortest distance to the stopped user appendix is likely to be the vertex that the user touches or points to.
    At block 1314, once the first true vertex has been identified, the user can be instructed to place the tag on a second corner of the box object, and the second vertex of the box object can be identified in a similar manner as method 1300 repeats. This method can be repeated until all corner points of the box object have been identified or until a number
    BE2018 / 5780 corner points of the box object that is sufficient for dimensioning has been identified.
    As described above, with the coordinates of the real angles and the known size of the reference points (e.g. the size of a shipping label that is recognizable in the image data), the dimensions of the box object (e.g. the lengths of the edges) are determined, using, for example, the methods and devices described in US Patent No. 9,741,134.
    The benefits, solutions to problems and any element that can lead to any benefit, or solution that takes place or becomes more pronounced, should not be interpreted as a critical, necessary, or essential measure or element of any of all claims.
    In addition, in this document, relative terms such as first and second, top and bottom, and the like can only be used to distinguish one entity or action from another entity or action without necessarily having any of these actual relationships or orders between such entities or actions is needed or is implied. The terms "includes", "comprising", "" has "," having "," contains "," containing "or any other variation thereof are intended to cover a non-exclusive inclusion such that a process, process, article, or device that includes a list of elements, includes not only those elements, but may also include other elements that are not explicitly mentioned or inherent in such a process, method, article, or device. An element preceded by "includes ... a", "has ... a", "contains ... a" does not exclude, without limitation, the existence of additional identical elements in the process, method, article , or the device comprising the element. The term "one" is defined as one or more, unless explicitly stated otherwise herein. The terms "essential", "essential", "approximately", or any
    BE2018 / 5780 other version thereof, are defined as being close by, as understood by those skilled in the art, and in a non-limiting embodiment, the term is defined as being within 10%, in another embodiment as being within 5%, in another embodiment as being within 1% and in another embodiment as being within 0.5%. The term "coupled" as used herein is defined as connected, although not necessarily directly and not necessarily mechanically. A device or structure that is "configured" in a certain way is configured in at least that way, but can also be configured in ways that are not specified.
    Some embodiments may include one or more generic or specialized processors (or "processing devices") such as microprocessors, digital signal processors, customized processors, and field programmable gate arrays (FPGAs) and uniquely stored program instructions (including both software and hardware) that are the one or control multiple processors, in combination with certain non-processor circuits, to implement some, most, or all functions of the method and / or device described herein. Alternatively, some or all of the functions could be implemented by a state machine that has no stored program instructions, or in one or more application-specific integrated circuits (ASICs), in which each function or some combinations of certain functions are implemented as custom logic ( tailored). Of course, a combination of the two approaches could be used.
    In addition, an embodiment may be implemented as a computer-readable storage medium with a computer-readable code stored thereon for programming a computer (e.g., including a processor) to perform a method as described and claimed herein. Examples of such computer-readable storage media include, but are not
    BE2018 / 5780 limited to, a hard disk, a CD-ROM, an optical storage device, a magnetic storage device, a ROM (read-only memory), a PROM (programmable read-only memory), an EPROM (erasable programmable read-only) memory), an EEPROM (electrically erasable programmable read-only memory) and a flash memory. Those skilled in the art, despite potentially significant effort and many design choices motivated by, for example, available time, current technology, and economic considerations, when guided by the concepts and principles described herein will be easily able to generate such software instructions. and programs and ICs with minimal experimentation.
    权利要求:
    Claims (20)
    [1]
    A computer-implemented method, comprising:
    recording, by a plurality of cameras, image data representative of an object;
    analyzing, by a processor, the image data to identify a plurality of candidate angles of the object;
    detecting, by the processor, a proximity of an appendix to each of the candidate angles;
    confirming, by the processor, based on the respective proximity of the tag to the candidate corners of the object, that a first of the candidate corners is an angle of the object; and calculating, by the processor, based on the confirmed angle of the object, a size of the object.
    [2]
    The computer-implemented method of claim 1, wherein analyzing the image data to identify the candidate angles of the object comprises:
    identifying, by the processor, portions of the image data indicative of one or more edges of the object;
    extrapolating, by the processor, the one or more edges of the object to determine a multiple number of intersections of the edges; and identifying, by the processor, the candidate angles of the object based on the intersections of the edges.
    [3]
    The computer-implemented method of claim 1 or 2, wherein detecting a proximity of the tag to one of the
    BE2018 / 5780 candidate corners of the object and confirming that the first of the candidate corners is an angle:
    analyzing the image data to identify portions of the image data indicative of the tag;
    detecting that the tag has stopped moving for a threshold period;
    determining a distance between the stopped appendix and each of the candidate angles; and identifying the first of the candidate angles if it has the shortest distance to the stopped appendage of the user.
    [4]
    The computer-implemented method of claim 3, further comprising:
    instructing a user to place the tag on a first corner of the object prior to determining the distance between the stopped tag and each of the candidate angles.
    [5]
    The computer-implemented method according to claim 4, further comprising:
    after confirming the angle of the object, instructing the user to place the appendage at a second angle of the object.
    [6]
    The computer-implemented method of any one of the preceding claims, further comprising:
    detecting, by a sensor, a signal transmitted from a transmitter associated with the user's tag, the signal being generated in response to a user's tactile action.
    [7]
    The computer-implemented method of claim 6, wherein detecting the proximity includes:
    BE2018 / 5780 detecting a first proximity of the tag to one of the candidate angles in response to detecting the signal transmitted by the transmitter associated with the tag.
    [8]
    The computer-implemented method of any one of the preceding claims, wherein calculating the size of the object comprises:
    determining a length of an edge of the object.
    [9]
    The computer-implemented method of any one of the preceding claims, wherein:
    calculating the size of the object analyzing image data to identify a reference mark associated with the object, the reference mark having a known size; and calculating the size of the object is based on the known size of the reference mark.
    [10]
    The computer-implemented method of claim 9, wherein calculating the size of the object comprises:
    determining, based on the angle of the object, a length of each edge of the box object; and determining, using the lengths of the edges, a volume of the object.
    [11]
    11. System comprising:
    a heads-up display assembly comprising a presentation generator and a head mount, the heads-up display assembly being configured to be worn by a user;
    a plurality of cameras attached to the heads-up display assembly, the cameras being configured to record image data associated with an object and an attachment of the user;
    BE2018 / 5780 a memory configured to store computer executable instructions; and a processor configured to interact with the presentation generator, the plurality of cameras, and the memory, and configured to execute the computer-executable instructions to turn the processor on:
    analyzing the image data associated with the object to identify a plurality of candidate angles of the object;
    detecting a proximity of a user's tag to a first of the candidate corners of the object;
    determining, based on the proximity of the appendix to the first of the candidate angles, whether the first of the candidate angles is a true angle of the object; and when the first of the candidate angles is a real angle of the object, calculating a size of the object based on the first of the candidate angles.
    [12]
    The system of claim 11, wherein the computer-executable instructions cause the processor, when executed, to identify the candidate angles by:
    identifying data indicative of one or more edges of the box object;
    extrapolating the one or more edges of the object; and identifying the candidate angles of the object based on intersections of the extrapolated edges.
    [13]
    The system of claim 11 or 12, wherein the computer-executable instructions cause the processor to detect the proximity of the tag to the first of the candidate corners by:
    BE2018 / 5780 detecting that the tag has stopped moving for a threshold time period; and in response to detecting that the tag has stopped moving for a threshold time period, determining a distance between the stopped tag of the user and the first of the candidate angles.
    [14]
    The system of claim 13, wherein the computer-executable instructions prompt the processor to:
    instructing the user to place the tag on a first corner of the object prior to determining the distance between the stopped tag and the first of the candidate angles.
    [15]
    The system of claim 14, wherein the computer-executable instructions prompt the processor to:
    instructing, after determining that the first of the candidate angles is a real angle of the object, the user to place the tag on a second corner of the object.
    [16]
    The system of any one of the preceding claims 11-15, further comprising a sensor, wherein the computer-executable instructions cause the processor to detect, via the sensor, a signal transmitted by a transmitter carried by the user, the signal being generated in response to a tactile action by the user.
    [17]
    The system of claim 16, wherein the computer-executable instructions cause the processor to detect the proximity of the tag to the first of the candidate angles of the object in response to the signal transmitted by the transmitter.
    [18]
    The system of claim 16 or 17, wherein the computer-executable instructions set the processor to calculate the
    BE2018 / 5780 dimension of the object by determining a length of an edge of the object.
    [19]
    The system of claim 18, wherein:
    the computer-executable instructions turn on the processor
    5 to analyze the image data to identify a reference mark associated with the object, the reference mark having a known size; and the computer-executable instructions prompt the processor to calculate the size of the object based on the known
    10 dimension of the reference mark and the determined length of the edge.
    [20]
    The system of any one of the preceding claims 11-19, wherein the computer-executable instructions cause the processor to calculate the size of the object by:
    determining a length of an edge of the object; and
    Determining, using the length of the edge of the object, a volume of the object.
    类似技术:
    公开号 | 公开日 | 专利标题
    JP6951056B2|2021-10-20|Acceleration-based motion tolerance and predictive coding
    US9846966B2|2017-12-19|Image processing device, image processing method, and computer program product
    US20210082198A1|2021-03-18|Augmented reality lighting effects
    US10249030B2|2019-04-02|Image transformation for indicia reading
    CN110266916B|2021-09-21|Method and system for processing glare in eye tracking
    EP2779092A1|2014-09-17|Apparatus and techniques for determining object depth in images
    KR102322813B1|2021-11-08|3d silhouette sensing system
    US20160104274A1|2016-04-14|Image-stitching for dimensioning
    US20130127705A1|2013-05-23|Apparatus for touching projection of 3d images on infrared screen using single-infrared camera
    US20140355838A1|2014-12-04|Recognition dictionary creation apparatus and method for creating recognition dictionary by the same
    US10642422B2|2020-05-05|Information processing apparatus, control method for the information processing apparatus, and storage medium
    CN108027656B|2021-07-06|Input device, input method, and program
    US20180096332A1|2018-04-05|A pos terminal
    BE1025916B1|2020-02-07|METHODS AND DEVICES FOR QUICK DIMENSIONING AN OBJECT
    EP3158921A1|2017-04-26|Line of sight detection system and method
    BE1025917B1|2020-02-07|METHODS AND DEVICES FOR DIMENSIONING AN OBJECT USING NEAR DEVICES
    JP2017067503A|2017-04-06|Position estimation system, position estimation method, and position estimation program
    WO2017179543A1|2017-10-19|Information processing device, information processing method, and program recording medium
    JP2018066823A|2018-04-26|Information display device and method for controlling processing thereof
    US10621398B2|2020-04-14|Methods and systems for operating an indicia scanner
    US20200201513A1|2020-06-25|Systems and methods for rfid tag locationing in augmented reality display
    JPWO2015145977A1|2017-04-13|Information processing apparatus, image processing method and program, and POS terminal apparatus
    JP6388108B2|2018-09-12|POS terminal device, POS system, information processing system, image recognition method, and image recognition program
    KR20200074050A|2020-06-24|Identification devices and electronics
    JP2016143199A|2016-08-08|Input device, input method, computer program for input processing, and input system
    同族专利:
    公开号 | 公开日
    CN111316320B|2021-06-15|
    GB2581094A|2020-08-05|
    CN111316320A|2020-06-19|
    US10621746B2|2020-04-14|
    BE1025916A1|2019-08-07|
    GB202006159D0|2020-06-10|
    DE202018006634U1|2021-09-24|
    GB2581094B|2021-02-17|
    WO2019094164A1|2019-05-16|
    US20190139250A1|2019-05-09|
    DE112018005311T5|2020-06-18|
    引用文献:
    公开号 | 申请日 | 公开日 | 申请人 | 专利标题
    US9741134B2|2013-12-16|2017-08-22|Symbol Technologies, Llc|Method and apparatus for dimensioning box object|
    US20160147408A1|2014-11-25|2016-05-26|Johnathan Bevis|Virtual measurement tool for a wearable visualization device|
    US20170017301A1|2015-07-16|2017-01-19|Hand Held Products, Inc.|Adjusting dimensioning results using augmented reality|
    WO2008124107A1|2007-04-04|2008-10-16|The Regents Of The University Of California|Compositions, devices, systems, and methods for using a nanopore|
    US9841311B2|2012-10-16|2017-12-12|Hand Held Products, Inc.|Dimensioning system|
    US10133342B2|2013-02-14|2018-11-20|Qualcomm Incorporated|Human-body-gesture-based region and volume selection for HMD|
    US9233470B1|2013-03-15|2016-01-12|Industrial Perception, Inc.|Determining a virtual representation of an environment by projecting texture patterns|
    US9747696B2|2013-05-17|2017-08-29|Leap Motion, Inc.|Systems and methods for providing normalized parameters of motions of objects in three-dimensional space|
    US9494415B2|2013-11-07|2016-11-15|Intel Corporation|Object position determination|
    US9799111B2|2016-02-11|2017-10-24|Symbol Technologies, Llc|Methods and systems for highlighting box surfaces and edges in mobile box dimensioning|
    US10089750B2|2017-02-02|2018-10-02|Intel Corporation|Method and system of automatic object dimension measurement by using image processing|US10885653B2|2018-01-12|2021-01-05|United Parcel Service Of America, Inc.|Systems and methods for mobile parcel dimension calculation and predictive condition analysis|
    法律状态:
    2020-04-02| FG| Patent granted|Effective date: 20200207 |
    优先权:
    申请号 | 申请日 | 专利标题
    US15/806,203|US10621746B2|2017-11-07|2017-11-07|Methods and apparatus for rapidly dimensioning an object|
    [返回顶部]